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3. (original) The method of Claim 1, wherein the voice recognition 
information further comprises a confidence threshold associated with the recognized 
voice command. 

4. (original) The method of Claim 1, wherein the voice recognition 
information further comprises a recognition delay associated with the recognized voice 
command. 

5. (original) The method of Claim 1, wherein the transcribed utterance is 
recorded in a transcription log in association with the recognized voice conmiand and the 
corresponding voice recognition information. 

6. (original) The method of Claim 5, wherein the voice recognition 
information includes the audio file that includes the user utterance. 

7. (original) The method of Claim 5, further comprising: 

analyzing information recorded in the transcription log to detect a type of 
recognition error. 

8. (currently amended) The method of Claim 7, further comprising: 
determining ^ a source of the detected recognition error based on the 

type of recognition error. 



9. 



(original) The method of Claim 8, further comprising: 
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hypothesizing a solution to eliminate the source of error. 

10. {original) The method of Claim 5, further comprising: 

determining recognition accuracy based on the information recorded in 
the transcription log. 

11. {currently amended) A system for monitoring recognition accuracy of a 
voice recognition system having at least one recognition grammar comprising: 

software for retrieving voice recognition information stored in a call log 
by a voice recognition system in response to recognizing a user utterance, the voice 
recognition information comprising a recognized voice command associated with the 
user utterance and a reference to an audio file that includes the user utterance; 

a transcription log produced by the transcription software, the 
transcription log comprising an entry associated with the user utterance, the entry 
comprising: 

the recognized voice command; and 

a transcribed utterance produced fi-om transcribing the user 
utterance, if the user utterance included in the audio file does not match the recognized 
voice command associated with it; 

wherein the software determines a recognition accuracy for the voice 
recognition system by analyzing the entry recorded in the transcription log. 
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12. (original) The system of Claim 11, wherein the voice recognition 
information further comprises information for a recognition grammar used to recognize 
the user utterance. 

13. (original) The system of Claim 11, wherein the voice recognition 
information further comprises a confidence threshold associated with the recognized 
voice conmiand. 

14. (original) The system of Claim 11, wherein the voice recognition 
information further comprises: 

a recognition delay associated with the recognized voice command 

15. (original) The system of Claim 11, wherein the transcribed utterance is 
compared with the recognized voice command to determine if an error in recognition has 
occurred. 

16. (original) The system of Claim 15, wherein the software records in an 
error log a type of recognition error associated with a recognized user utterance, if the 
transcribed utterance does not match the recognized voice command. 

17. (currently amended) The system of Claim 16, wherein the software 
records an in-grammar false reject (IGFR) type error, if the transcribed utterance is part 
of the recognition grammar but the user utterance is not recognized bv the voice 
recognition svstem as part of the recognition grammar . 
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18. {original) The system of Claim 16, wherein the software records an in- 
grammar false accept (IGF A) type error, if the transcribed utterance is part of the 
recognition grammar and the recognized voice conmiand does not match the transcribed 
utterance. 

19. (currently amended) The system of Claim 16, wherein the software 
records an out-of-grammar false accept (OGFA) type error, if the transcribed utterance is 
not part of the recognition grammar and th e r e cogniz e d voic e command but the user 
utterance is mistakenly match e d with th e transcrib e d utt e ranc e recognized by the voice 
recognition system as the recognized voice command . 

20. {original) The system of Claim 16, wherein the software records an out- 
of-grammar correct reject (OGCR) type error if the transcribed utterance is not part of 
the recognition grammar but the transcribed utterance is frequently recorded in the 
transcription log. 

21 . {currentiy amended) The system of Claim [[17]] 18, wherein: 

a source of error is determined to be a low confidence threshold, if an 
IGFA type error is detected. 

22. {currently amended) The system of Claim [[17]]i8, wherein: 

a source of error is determined to be acoustic similarity between the recognized 
voice command and the user utterance, if an IGFA type error is detected. 
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23. {currently amended) The system of Claim [[17]]i8, wherein the 
recognition grammar includes one or more acoustic models for [[a]] the recognized voice 
command, wherein: 

a source of error is determined to be lack of sufficient acoustic models in 
the recognition grammar, if an IGF A type error is detected. 

24. {currently amended) The system of Claim [[18]]i7, wherein a source of 
error is determined to be a high confidence threshold, if an IGFR type error is detected. 

25. {currently amended) The system of Claim [[18]] 1/7, wherein the 
recognition grammar includes one or more acoustic models for [[a]] the recognized voice 
command, wherein: 

a source of error is determined to be lack of sufficient acoustic models in 
the recognition grammar, if an IGFR type error is detected. 

26. {currently amended) The system of Claim [[18]] 17, wherein the voice 
recognition system is configured with a first recognition sensitivity threshold, wherein: 

a source of error is determined to be a low sensitivity threshold, if an 
IGFR type error is detected. 

27. {original) The system of Claim 19, wherein: 

a source of error is determined to be acoustic similarity between the 
recognized voice command and the user utterance, if an OGFA type error is detected. 
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28. {original) The system of Claim 19, wherein the voice recognition system 
is configured with a first recognition sensitivity threshold, wherein: 

a source of error is determined to be a high sensitivity threshold, if an 
OGFA type error is detected. 

29. {original) The system of Claim 19, wherein a source of error is 
determined to be a low confidence threshold, if an OGFA type error is detected. 



30. {original) The system of Claim 20, wherein the recognition grammar 
includes one or more acoustic models for the recognized voice command, wherein: 

a source of error is determined to be lack of sufficient acoustic models in 
the recognition grammar, if an OGCR type error is detected. 



31. {original) A method for automatically improving a voice recognition 
system comprising: 

determining voice recognition accuracy by analyzing voice recognition 
information produced by a voice recognition system in response to receiving a user 
utterance, the voice recognition system comprising a recognition grammar and a first 
configuration, the voice recognition information comprising a recognized voice 
command associated with the user utterance; 

detecting a recognition error, if the recognized voice command does not 
match a transcribed utterance for the user utterance; 

determining at least one source of error for the detected recognition error; 



Applicant: Sean Doyle 
Appl. No. 10/023,914 

hypothesizing at least one solution to eliminate the source of error; and 

modifying the first configuration based on the hypothesized solution to 

implement a second configuration for the voice recognition system with better 

recognition accuracy. 

32. {original) The method of Claim 3 1 , fiirther comprising: 

testing the voice recognition system to detect improvements in voice 
recognition accuracy; and 

restoring the first configuration, if voice recognition accuracy is not 

improved. 

33. {original) The method of Claim 32, wherein the voice recognition 
information fiirther comprises information for a recognition grammar used to recognize 
the user utterance. 

34. {original) The method of Claim 33, wherein the voice recognition 
information fiirther comprises a confidence threshold associated with the recognized 
voice command. 

35. {original) The method of Claim 34, wherein the voice recognition 
information fiirther comprises a recognition delay associated with the recognized voice 
command. 



-10- 
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36. (original) The method of Claim 35, wherein an in-grammar false accept 
(IGF A) type error is detected, if the transcribed utterance is part of the recognition 
grammar and the recognized voice command does not match the transcribed utterance. 

37. {currently amended) The method of Claim 35, wherein an in-grammar 
false reject (IGFR) type error is detected, if the transcribed utterance is part of the 
recognition grammar but the user utterance is not recognized by the voice recognition 
system as part of the recognition grammar . 

38. {currently amended) The method of Claim 35, wherein an out-of- 
grammar false accept (OOP A) type error is detected, if the transcribed utterance is not 
part of the recognition grammar and th e recogniz e d voice command but the user 
utterance is mistakenly match e d with th e transcrib e d utt e ranc e recognized by the voice 
recognition system as the recognized voice command . 

39. {currently amended) The method of Claim 35, wherein an out-of- 
grammar correct reject (OGCR) type error is detected, if the transcribed utterance is not 
part of the recognition grammar but the transcribed utterance is frequently recorded-in 
th e transcription log . 

40. {original) The method of Claim 36, wherein a source of error is 
determined to be a low confidence threshold, if an IGFA type error is detected. 
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41. {original) The method of Claim 36, wherein a source of error is 
determined to be acoustic similarity between the recognized voice command and the user 
utterance, if an IGFA type error is detected. 

42. {currently amended) The method of Claim 36, wherein the recognition 
grammar includes one or more acoustic models for [[a]] the recognized voice command, 
wherein a source of error is determined to be lack of sufficient acoustic models in the 
recognition grammar, if an IGFA type error is detected. 

43. {original) The method of Claim 37, wherein a source of error is 
determined to be a high confidence threshold, if an IGFR type error is detected. 

44. {currently amended) The method of Claim 37, wherein the recognition 
grammar includes one or more acoustic models for [[a]] tiie recognized voice command, 
wherein a source of error is determined to be lack of sufficient acoustic models in the 
recognition grammar, if an IGFR type error is detected. 

45. {original) The method of Claim 37, wherein the voice recognition system 
is configured with a first recognition sensitivity threshold, wherein a soiirce of error is 
determined to be a low sensitivity threshold, if an IGFR type error is detected. 

46. {original) The method of Claim 38, wherein a source of error is 
determined to be acoustic similarity between the recognized voice command and the user 
utterance, if an OGFA type error is detected. 
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47. {original) The method of Claim 38, wherein the voice recognition system 
is configured with a first recognition sensitivity threshold, wherein a source of error is 
determined to be a high sensitivity threshold, if an OGFA type error is detected. 

48. {original) The method of Claim 38, wherein a source of error is 
determined to be a low confidence threshold, if an OGFA type error is detected. 

49. {currently amended) The method of Claim 39, wherein the recognition 
grammar includes one or more acoustic models for [[a]] flie recognized voice command, 
wherein a source of error is determined to be lack of sufficient acoustic models in the 
recognition grammar, if an OGFR OGCR type error is detected. 

50. {original) A method of improving recognition accuracy and efficiency in 
a voice recognition system having a grammar, wherein a received utterance is accepted if 
the utterance is an acoustic match with at least one term included in the grammar and 
rejected otherwise, the method comprising: 

determining if the utterance has been falsely accepted by the voice 
recognition system; and 

removing a term firom the grammar, if the utterance is part of the grammar 
but is falsely accepted as another term included in the grammar due to acoustic 
similarity. 

5 1 . {original) The method of Claim 50, fiirther comprising: 
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adding to the grammar one or more synonyms which are acoustically 
dissimilar to the utterance if it has been falsely accepted. 

52. {original) The method of Claim 50, wherein the voice recognition system 
comprises at least one acoustic model for the utterance, the method further comprising: 

adding to the voice recognition system an additional acoustic model for 
the utterance, if the utterance is part of the grammar, but is falsely rejected due to lack of 
sufficient altemative acoustic models. 

53. {original) The method of Claim 52, wherein the additional acoustic model 
is one that is associated with the rejected utterance. 

54. {original) The method of Claim 50, wherein the voice recognition system 
has a confidence threshold for adjusting recognition accuracy based on degree of 
acoustic similarity between the utterance and one or more acoustic models included in 
the voice recognition system, wherein the recognition accuracy is improved if the 
confidence threshold is higher, the method fiuther comprising: 

lowering the confidence threshold, if the utterance is part of the grammar, 
but is falsely rejected due to a high confidence threshold. 

55. {currently amended) The method of Claim 50, wherein the voice 
recognition system having has a sensitivity threshold for accepting utterances, wherein a 
higher sensitivity allows for a larger number of utterances to be considered by the voice 
recognition system, the method fiirther comprising: 
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lowering the sensitivity threshold if a predetermined amount of noise is 
undesirably considered by the voice recognition system. 

56. {original) The method of Claim 55, further comprising: 

increasing the sensitivity threshold, if the utterance is part of the grammar, 
but is falsely rejected due to a low sensitivity threshold. 

57. {currently amended) The method of Claim 50, wherein the voice 
recognition system having has a timeout threshold and an end-of-speech threshold for 
accepting an utterance having a certain audio length or audio delay, the method further 
comprising: 

adjusting the timeout and the end-of-speech thresholds, if a predetermined 
amount of noise is undesirably considered for acceptance by the system. 

58. {original) The method of Claim 50, wherein the grammar has a perplexity 
associated with a nimiber of decision levels and terms included in the granmiar, the 
method further comprising: 

reducing grammar perplexity, if recognition of the utterance is delayed 
beyond a threshold value. 

59. {original) The method of Claim 50, further comprising: 

monitoring frequently received utterances that are rejected because they 
are not part of the grammar; and 

adding the frequently received utterances to the grammar. 
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60. {original) A method performed on a computer for monitoring recognition 
accuracy in a voice recognition system comprising: 

recording a user utterance in an audio file; 

storing a voice command which was recognized by the voice recognition 
system for the user utterance; 

presenting the audio file and the voice command to an operator for 

selection; 

playing the user utterance firom the audio file for the operator upon 
selection by the operator; 

receiving a transcription of the user utterance; and 

associating the transcription with the recognized voice command. 
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